Supporting Efficient Noncontiguous Access in PVFS over InfiniBand
نویسندگان
چکیده
Noncontiguous I/O access is the main access pattern in many scientific applications. Noncontiguity exists both in access to files and in access to target memory regions on the client. This characteristic imposes a requirement of native noncontiguous I/O access support in cluster file systems for high performance. In this paper, we address two main issues on supporting efficient noncontiguous I/O access in cluster file systems over a high performance network. One is noncontiguous data transmission between the client and the I/O server. The second is noncontiguous disk access on the I/O server itself. We propose a novel approach, RDMA Gather/Scatter, to transfer noncontiguous data for such I/O accesses, and design a new scheme, Optimistic Group Registration, to reduce memory registration costs associated with this approach. For the second issue, we deploy data sieving on the I/O server to process a large number of small noncontiguous disk accesses. Unlike other data sieving implementations, our I/O server uses a cost model actively and intelligently to decide whether it is beneficial to perform data sieving or not. We have designed and incorporated these approaches in a version of PVFS over InfiniBand. Through a range of PVFS and MPI-IO micro-benchmarks, the MPI-IO tiled access test and the NAS BTIO benchmark, we demonstrate that our approaches attain significant performance gains compared to other existing approaches.
منابع مشابه
Benefits of Quadrics Scatter / Gather to PVFS 2
Noncontiguous IO access is the main access pattern in scientific applications. We have designed an algorithm that supports zerocopy noncontiguous PVFS2 IO using a software scatter/gather mechanism over Quadrics. To investigate what impact Quadrics scatter/gather mechanism can have on PVFS2 IO accesses, in this paper, we perform an indepth evaluation of the scatter/gather mechanism. We also stud...
متن کاملts of Quadrics Scatter / Gather to PVFS 2
Noncontiguous IO access is the main access pattern in scientific applications. We have designed an algorithm that supports zerocopy noncontiguous PVFS2 IO using a software scatter/gather mechanism over Quadrics. To investigate what impact Quadrics scatter/gather mechanism can have on PVFS2 IO accesses, in this paper, we perform an indepth evaluation of the scatter/gather mechanism. We also stud...
متن کاملPVFS over InfiniBand: Design and Performance Evaluation
I/O is quickly emerging as the main bottleneck limiting performance in modern day clusters. The need for scalable parallel I/O and file systems is becoming more and more urgent. In this paper, we examine the feasibility of leveraging InfiniBand technology to improve I/O performance and scalability of cluster file systems. We use Parallel Virtual File System (PVFS) as a basis for exploring these...
متن کاملEvaluating structured I/O methods for parallel file systems
Modern data-intensive structured datasets constantly undergo manipulation and migration through parallel scientific applications. Directly supporting these time consuming operations is an important step in providing high-performance I/O solutions for modern large-scale applications. High-level interfaces such as HDF5 and Parallel netCDF provide convenient APIs for accessing structured datasets,...
متن کاملNoncontiguous I/O Accesses Through MPI-IO
I/O performance remains a weakness of parallel computing systems today. While this weakness is partly attributed to rapid advances in other system components, I/O interfaces available to programmers and the I/O methods supported by file systems have traditionally not matched efficiently with the types of I/O operations that scientific applications perform, particularly noncontiguous accesses. T...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2003